Papers with open-ended text generation
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing tools for text-to-image synthesis can visualize machine imaginations for a given context. |
| Approach: | They propose a framework that uses machine-generated images to guide language models in open-ended text generation. |
| Outcome: | The proposed framework is effective on open-ended text generation tasks while showing minor degeneration. |
Penalty Decoding: Well Suppress the Self-Reinforcement Effect in Open-Ended Text Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate the efficacy of our approach in generating high-quality sentences resembling human output. |
| Approach: | They propose a forgetting mechanism that disregards distant tokens, reducing the burden of penalty selection. |
| Outcome: | The proposed approach generates high-quality sentences resembling human output. |
Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization (2023.findings-eacl)
Copied to clipboard
Aishwarya Agrawal, Ivana Kajic, Emanuele Bugliarello, Elnaz Davoodi, Anita Gergely, Phil Blunsom, Aida Nematzadeh
| Challenge: | Visual question answering (VQA) is a task of answering open-ended questions about images. |
| Approach: | They evaluate two vision-and-language (V&L) models under different settings . they find they tend to learn to solve the benchmark rather than the skills required by VQA . |
| Outcome: | The proposed models exhibit poor generalization under out-of-distribution settings. |
Evaluating Text Generation Quality Using Spectral Distances of Surprisal (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing metric fails to capture text surprisal, but FACE-2 produces stronger agreement with human preferences. |
| Approach: | They propose a new automatic evaluation metric for open-ended text generation . they propose metric that extracts the dynamic patterns (spectrum) of text surprisal . |
| Outcome: | The proposed metric outperforms existing methods in revealing the model scaling effect . it produces stronger agreement with human preferences from a large human-annotated dataset . |
Semantic Agreement Enables Efficient Open-Ended LLM Cascades (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Large language models (LLMs) have enabled impressive progress across a range of language tasks, but they are steep computational cost. |
| Approach: | They propose a semantic consensus mechanism for reliable deferral by combining model outputs with semantic consensus. |
| Outcome: | Evaluated from 500M to 70B-parameter models, semantic cascades improve deferral accuracy, match or surpass target-model quality at 40% of the cost and reduce latency by up to 60%. |
Towards Informative Open-ended Text Generation with Dynamic Knowledge Triples (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models (PLMs) have impressive capabilities in open-ended text generation. |
| Approach: | They propose a dynamic knowledge-guided informative open-ended text generation approach that utilizes a knowledge graph to help the model generate more contextually related entities and detailed facts. |
| Outcome: | The proposed approach generates more informative texts than baselines. |
Facts2Story: Controlling Text Generation by Key Facts (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for story generation struggle with staying coherent for long periods of time. |
| Approach: | They propose a controlled generation task which expands a sequence of facts into a longer narrative. |
| Outcome: | The proposed model produces competitive fluency while adhering to the requested facts. |
Event Transition Planning for Open-ended Text Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | Open-ended text generation tasks require models to generate coherent continuation given limited preceding context. |
| Approach: | They propose a novel two-stage method which explicitly arranges ensuing events in open-ended text generation tasks. |
| Outcome: | The proposed method improves coherence and diversity of open-ended text generation tasks. |
LexTempus: Enhancing Temporal Generalizability of Legal Language Models Through Dynamic Mixture of Experts (2025.acl-long)
Copied to clipboard
| Challenge: | a rapid evolution of legal concepts requires that legal language models adapt swiftly accounting for the temporal dynamics. |
| Approach: | They propose a dynamic mixture of experts model that explicitly models the temporal evolution of legal language in an online learning framework. |
| Outcome: | The proposed model can model the temporal evolution of legal language without forgetting past knowledge. |
Fine-Grained Controllable Text Generation Using Non-Residual Prompting (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to control the text generation process are not expressive enough. |
| Approach: | They propose an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps. |
| Outcome: | The proposed architecture is expressive and versatile on multiple experimental settings. |
Explaining and Improving Contrastive Decoding by Extrapolating the Probabilities of a Huge and Hypothetical LM (2024.emnlp-main)
Copied to clipboard
| Challenge: | Contrastive decoding (CD) improves the next-token distribution of a large expert language model (LM) using a small amateur LM. |
| Approach: | They propose a new unsupervised decoding method called Asymptotic Probability Decoding (APD) that extrapolates the probability curves from the LMs of different sizes to infer the asymptototic probabilities from an infinitely large LM. |
| Outcome: | The proposed method improves the next-token distribution of a large expert language model using a small amateur LM. |
Evaluating Large Language Model Biases in Persona-Steered Generation (2024.findings-acl)
Copied to clipboard
| Challenge: | a recent wave of powerful new large language models has raised concerns that their expressed opinions may be biased towards certain political, national or moral viewpoints. |
| Approach: | They define an incongruous persona as a persona with multiple traits where one trait makes its other traits less likely in human survey data. |
| Outcome: | The results show that LLMs are less steerable towards incongruous personas than congruous ones . the models that are fine-tuned with RLHF are more steerable, especially towards stances associated with political liberals and women . |
ModeX: Evaluator-Free Best-of-N Selection for Open-Ended Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to selecting a single high-quality output from multiple generations are limited in their applicability and efficiency. |
| Approach: | They propose a method that generalizes majority voting to open-ended text generation . modeX leverages structural information across multiple generation paths to select a "modal" output . |
| Outcome: | The proposed framework outperforms standard single- and multi-path baselines in open-ended tasks. |
UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing metrics for open-ended text generation are poorly correlated with human judgments . despite the success of existing metrics, there are few plausible outputs for the same input . |
| Approach: | They propose a UNreferenced measure for evaluating open-ended story generation . it is built on top of BERT and is trained to distinguish human-written stories from negative samples . |
| Outcome: | The proposed measure is more generalizable than state-of-the-art metrics and correlates better with human judgments. |
Triviality Corrected Endogenous Reward (2026.acl-long)
Copied to clipboard
Xinda Wang, Zhengxu Hou, Yangshijie Zhang, null Yanbingren, Jialin Liu, ChenZhuo Zhao, Zhibo Yang, Bin-Bin Yang, Feng Xiao
| Challenge: | Recent work on unsupervised reinforcement learning for mathematical reasoning using confidence-based endogenous rewards focuses on open-ended text generation, requiring either annotated data or powerful closed-source models. |
| Approach: | They propose a method that rewards the relative information gain between a specialist and a generalist reference policy, modulated by a probability-dependent correction mechanism. |
| Outcome: | The proposed model improves on multiple writing benchmarks and model architectures without external supervision and validates generality across different generation tasks. |
MolCA: Molecular Graph-Language Modeling with Cross-Modal Projector and Uni-Modal Adapter (2023.emnlp-main)
Copied to clipboard
| Challenge: | Language Models (LMs) have demonstrated impressive molecule understanding ability on 1D text-related tasks, but lack 2D graph perception, a critical ability of human professionals in comprehending molecules’ topological structures. |
| Approach: | They propose to combine a cross-modal projector and a uni-modal adapter to enable an LM to understand both text- and graph-based molecular contents via a Q-Former. |
| Outcome: | The proposed model outperforms the baselines on tasks such as molecule captioning, IUPAC name prediction, and molecule-text retrieval. |
Factuality of Large Language Models: A Survey (2024.emnlp-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Georgiev, Rocktim Das, Preslav Nakov
| Challenge: | Large language models (LLMs) are factually incorrect, which limits their applicability in real-world scenarios. |
| Approach: | They analyze existing work to identify major challenges and their associated causes . they propose to evaluate LLMs using a variety of measures to mitigate factual errors . |
| Outcome: | The proposed methods are based on a variety of datasets and proposed strategies to mitigate factual errors. |
Odysseus Navigates the Sirens’ Song: Dynamic Focus Decoding for Factual and Diverse Open-Ended Text Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decoding methods struggle to balance factuality and diversity . Deterministic decoding approaches suffer from degeneration and lack of diversity - a problem that is not addressed by the current literature. |
| Approach: | They propose a plug-and-play stochastic approach that adjusts decoding focus based on distributional differences across layers, leveraging the modular nature of factual knowledge within LLMs. |
| Outcome: | Extensive experiments on seven datasets show that DFD significantly improves performance. |
MAGNET: Augmenting Generative Decoders with Representation Learning and Infilling Capabilities (2025.acl-long)
Copied to clipboard
| Challenge: | Decoder-only large language models are increasingly being adapted for bidirectional modeling . however, their reliance on causal attention restricts their effectiveness in tasks that require understanding of bidirectional context. |
| Approach: | They propose a method to adapt decoder-only large language models to generate robust representations and infill missing text spans. |
| Outcome: | The proposed method surpasses strong decoders on token-level and sentence-level representation learning tasks and generates contextually appropriate text infills without excessive repetition of words or phrases. |